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Membrane bioreactor employs an efficient filtration technology for solid and 
liquid separation in wastewater treatment process. Development of 
membrane filtration model is significant as this model can be used to predict 
filtration dynamic which is later utilized in control development. Most of the 
available models only suitable for monitoring purpose, which are too 
complex, required many variables and not suitable for control system design. 


Keyword: This work focusing on the simple time seris model for membrane filtration 
process using neural network technique. In this paper, submerged membrane 
ANN modeling filtration model developed using recurrent neural network (RNN) train using 
GA genetic algorithm (GA), inertia weight particle swarm optimization (IW- 
GSA PSO) and gravitational search algorithm (GSA). These optimization 
PSO algorithms are compared in term of its accuracy and convergent speed in 
SMBR updating the weights and biases of the RNN for optimal filtration model. The 
evaluation of the models is measured using three performance evaluations, 
which are mean square error (MSE), mean absolute deviation (MAD) and 
coefficient of determination (R2). From the results obtained, all methods 
yield satisfactory result for the model, with the best results given by 
IW-PSO. 
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1. INTRODUCTION 

Membrane filtration is a promising technology in many separation and purification process. This 
technology is able to achieve high performance in term of permeate quality. The application of membrane 
filtration is widely applied in many industrial applications such as water treatment and wastewater treatment 
process, food processing, desalination, medical application and many more. Despite the advance in the 
current technology, membrane filtration still experiencing numerous drawbacks in terms of fouling, high 
energy consumption and a significantly higher operational cost compared to conventional techniques [1]. 

To improve the membrane filtration process, one of the options is to obtain accurate modeling of the 
filtration process. With the obtained model, an accurate prediction can be made for future action of the 
process in order to improve the filtration performance of hydraulic cleaning involving aeration airflow, 
backwash, relaxation and chemical cleaning. From the obtained model, the best setting for the filtration 
permeate flux set point and the transmembrane pressure (TMP) dynamic behavior can also be determined. 
However, developing a working model for membrane filtration process is not an easy task, due to the 
nonlinear chaotic behavior of fouling. In literature, several mechanisms were developed to obtain an 
optimum filtration model. Darcy’s laws for example, several terms of fouling resistance are added in series to 
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represent the fouling layer in the membrane filtration process to develop the filtration model. Similar to other 
mechanistic or first principle model, tedious calibration process involving many parameters is required. 

An alternative technique to represent a dynamic behavior of the filtration process is using artificial 
neural network (ANN), which has high accuracy of prediction capability. The ANN technique is employed in 
numerous areas including chemical process, mechanical applications, financial forecasting and trending, 
environmental prediction and others. On the area of membrane filtration modeling, Geissler et al [2] has 
developed models which are semi empirical model and ANN based model for permeate flux modeling in 
submerged capillary MBR. The ANN model is based on Elman neural network structure to predict the 
permeate flux. The techniques yield very good results, where the semi empirical model required small input 
variables compared to ANN. However, the ANN model gave high accuracy with the average error of 2.7%. 

In [3], a flat sheet submerged membrane bioreactor (SMBR) filtration for wastewater filtration is 
modeled using ANN model to represent the backwash effect to the permeate flux. Multilayer feed forward 
neural network is used to model the system. Various backwashed and filtration interval are used as the input 
to the model, meanwhile flux is used as the output. With the given backwashed to permeate ratio, the model 
is able to predict the permeate flux performance. Nevertheless, the application of ANN using conventional 
back propagation (BP) algorithm for training has faced several problems such as slow convergent and the 
algorithm has tendency to tarp in the local minima. Thus, the application of heuristic search optimization 
technique is one of the solutions. 

Several works has found in literature to find an optimal weight and bias value in the training of the 
neural network. Among the widely used heuristic search algorithm for ANN model is genetic algorithm 
(GA).this algorithm is among the pioneer in heuristic optimization which is inspired from the population 
evolutionary theory. GA was successful used in [4], [5] and [6] to train the neural network model in various 
applications. This algorithm is able to produce a global solution for a given optimization problems. Particle 
swarms optimizations (PSO) in particular, have become a very effective optimization technique in various 
fields. The algorithm is fast and very reliable in searching for minimization. Several well known 
advancement of PSO algorithm such as random PSO (RPSO), constriction factor PSO (CPSO) and inertia 
weight PSO (IW PSO) have been developed throughout the years. These algorithms have been tested in 
previous neural networks application as reported in [7], [8] and [9]. Gravitational search algorithm (GSA) 
[10] is another type of optimization technique. This method is inspired by the law of gravity and the mass 
interaction in universe. The GSA was reported to yield faster convergent rate and is reliable to meet the 
objective function in many applications as reported in [10]-[12]. 

This work focuses on model development of membrane filtration process using recurrent neural 
network (RNN) model train by three type’s of soft computing optimization techniques which are GA, PSO 
and GSA. These three algorithms will be used to search for the best weights and biases of the recurrent neural 
network model. The trained mode is then compared in term of its accuracy on the training and testing of 
membrane filtration data set. 


2. EXPERIMENT SETUP 

The data set is collected from the developed membrane bioreactor pilot plant located in Process 
Control Lab, Faculty of Electrical Engineering, Universiti Teknologi Malaysia (UTM). Random magnitude 
steps input were given to the suction pump to stimulate the dynamic behavior of the process. Figure | shows 
the plant schematic diagram while Figure 2 shows the data collected from the pilot plant. The experiments 
were carried out in single tank submerged membrane bioreactors, with working volume of 20 L Palm Oil 
Mill Effluent (POME) taken from Sedenak Palm Oil Mill Sdn. Bhd. in Johor, Malaysia. The aeration during 
filtration is set around 6 to 8 litter per minute (LPM) to maintain the dissolved oxygen level to be more than 
2 ppm during the experiment. In this work, voltage input of the permeate pump is used for the input, mean 
while the flux and TMP are the measurement outputs. 


Neural Network Model Development with Soft Computing Techniques for ... (Norhaliza Abdul) 


2616 O ISSN: 2088-8708 


= 40 
g [™ in 
GPA , a m | 
a. m mmli |P a ray 
= gill Li 
0 500 1000 1500 2000 2500 3000 3500 4000 
Time (s) 
= 300 
so Fail e] 
5 / 7 ; P 
Æ 200 “| 7 P e F 7 
a A NETTA P al AA al Te An 
= oil im VW VE yy Ay VI U y y 
o 500 1000 1500 2000 2500 3000 3500 4000 
= Time (s) 
= 20 
= 
2 10 
3 yi nay a hull Fee es Wee ad penal Piry 
= o0 
=< o 500 1000 1500 2000 2500 3000 3500 4000 
Time (s) 
24 
5 
Z3 = [ 
PI-101 = al ie | 
5 Ui | 
rT Zo - 
Air Air Diffuser o 500 1000 1500 2000 2500 3000 3500 4000 
Time (s) 
Figure 1. Plant schamatic Figure 2. Experimental data 


In this work, Polyethersulfone (PES) material with approximately 80-100kda pore size and a surface 
area of approximately 0.35 m?is used in the filtration system 


3. MODEL DEVELOPMENT 

Recurrent neural network (RNN) model is a mathematical model developed based on the past input 
and past output of the system. The RNN is developed based on the basic feed forward neural network 
(FFNN). The training algorithm is employed to obtain suitable weights and biases of the network in order to 
minimize the error in the training procedure. Figure 3 shows the basic structure of the neural network model. 


yi (6) 


¥2(t) 


Figure 3. Neural network structure Figure 4. Recurrent neural network structure 


The neural network equation can be represented as: 
A n 
Ji@) = aee Wifi OMA Wij + Wjo) + Wio] (1) 


where ĵ;(t)is the prediction output. F; is the function of the network, @ is the input vector, Wj and w; 
represent the network connection layer weights and biases respectively. In this work three and five hidden 
neurons are tested with basic three layers neural networks which are the input layer, single hidden layer and 
output layer. This work employed nonlinear autoregressive with exogenous input (NARX) ANN structure 
which shows in Figure 4. The optimization techniques applied in the ANN model training is optimize the 
weights and biases of the ANN structure by minimizing the mean square error (MSE) of the model 


3.1. Genetic algorithm (GA) 
Optimization algorithm is a developing area in the soft computing study. GA algorithm is one of the 
pioneers in evolution based optimization techniques and is a reliable technique in optimization. This 
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optimization technique is widely applied in many industrial applications such as in [13] and [14]. The flow 
chart of the algorithm can be represented in Figure 5. 
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Figure 5. GA Optimization flowchart 


3.2. Particle swarm optimization 

Particle swarm optimization (PSO) is inspired by behavior of a group of animals hunting. This 
heuristic search optimization is very effective in finding optimal solution for many problems. In PSO 
algorithm, number of swarm must be selected to search for the solution. Each swarm contains an individual 
calls a particle. The PSO main algorithm is to update the position of each particle with an estimated velocity. 
Each of the components of the velocity equation represents the exploration ability and capability of 
individual learning as well as social learning. The PSO algorithm execution flow chart is shown in Figure 6. 
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Figure 6. PSO flowchart 


3.2. Gravitational search algorithm (GSA) 

The GSA algorithm is an effective optimization algorithm developed in [10]. It is inspired from 
gravity law and mass interaction in the universe. Figure 7 presents the flow chart of the GSA optimization 
algorithm. The details development of the algorithm is presented in [10]. 
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Figure 7. GSA flowchart 


4. RESULTS AND ANALYSIS 

Generally, all the training methods using soft computing techniques are able to produce acceptable 
result for the model obtained. From the training result it shows that PSO algorithm performs much better 
when compared with GA and GSA. From Figure 8(a) it can be observed that GA has a much faster searching 
for the right weights and biases of the neural network compared with the other. However, when approaching 
the first 50 iterations, PSO shows superiority in finding global minimum from the given cost function. 
Meanwhile, Figure 8(b) shows the zooming convergent area of the training algorithms. In term of modeling 
performance criteria, the PSO algorithm performs better compared with the other methods. Table 1(a) present 
the permeate flux training result. Table 1(b) shows the result for the TMP modeling. 
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Figure 8. (a) The performance of the training algorithm (b) Zoom at the convergent area of the training 
algorithm 


Table 1. (a) Permeate Flux Modeling Training Result 


Method R? MSE MAD 
GA-RNN 93.48 0.0066 0.0473 
PSO-RNN 94.38 0.0057 0.0418 
GSA-RNN 93.12 0.0070 0.0493 


Table 1. (o) TMP Modeling Training Result 
R? 


Method MSE MAD 
GA-RNN 97.90 0.0014 0.0287 
PSO-RNN 97.82 0.0015 0.0296 
GSA-RNN 83.59 0.0113 0.0848 


Figure 9 (a) and Figure 9(b) shows the comparison between actual and predicted training model using PSO- 
RNN for the permeate flux and TMP. 
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Figure 9. (a) Permeate Flux modeling training result using RNN-PSO three hidden neuron and the residuals 
between actual and model (b) TMP modeling training result using RNN-PSO three hidden neuron and the 
residuals between actual and model 


The validation of the model is done by using data not utilized for the model development. From the 
result given in the Table 2(a) and (b), it is proven that the training performances of all the algorithms are able 
to model the system. Generally, the testing result is less accurate compared with the training. However, from 
the testing performance it showed that the models are able to replicate the dynamic of the membrane 
filtration system as shown in Figure 10. 


Table 2. (a) Permeate Flux Modeling Testing Result 


Method %R? MSE MAD 
GA-RNN 93.62 0.0062 0.0452 
PSO-RNN 94.19 0.0056 0.0418 
GSA-RNN 92.76 0.0070 0.0448 


Table 2. (b) TMP Modeling Testing Result 


Method AR? MSE MAD 
GA-RNN 97.85 0.0013 0.0275 
PSO-RNN 97.87 0.0012 0.0283 
GSA-RNN 81.33 0.0115 0.0839 
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Figure 10. (a) Permeate Flux modeling testing result using RNN-PSO three hidden neuron and the residuals 
between actual and model (b) TMP modeling testing result using RNN-PSO three hidden neuron and the 
residuals between actual and model 
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The result of the increment of the hidden neurons of the neural network has shown significant 
difference from the initial (three hidden neurons) training. It is an interesting finding where some of the 
training algorithm shows improvement of the model while the other is not able to find suitable weights and 
biases of the neural network model until the end of the iterations. Figure 11(a) shows the performance for 
each of the training methods in term of it fitness and iterations. Figure 11(b) shows the zoomed image at the 
convergent area. 
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Figure 11. (a) The performance of the training algorithm for five hidden neuron (b) Zoom at the convergent 
area of the training algorithm for five hidden neuron 


The overall of the modeling performance for the training result is shows in Table 3(a) and 
Table 3(b). Permeate flux modeling had shown the PSO and GA can be used for the RNN training. 
Figure 12(a) and Figure 12(b) represents the training result for the permeate flux and TMP using the PSO- 
RNN modeling technique. 

Generally, the testing results have shown that the result is much improved compared with the three 
hidden neuron. The PSO and GA training algorithm are able to maintain the accuracy of the model with 
minimum error increment compared with the training dataset. The testing result reveal similar trend of 
accuracy with the training result where the PSO algorithm obtains higher accuracy of the model followed 
with GA and GSA. Table 4(a) and Table 4(b) present the testing performance of the models with the WR, 
MSE and MAD performance. The testing result for the permeate flux and TMP were shown in Figure 13(a) 
and Figure 13(b). 


Table 3. (a) Permeate Flux Modeling five hidden Neuron Training Result 


Method %R? MSE MAD 
GA-RNN 93.00 0.0071 0.0489 
PSO-RNN 95.26 0.0048 0.0321 
GSA-RNN 76.10 0.0242 0.0974 


Table 3. (b) TMP Modeling Training five hidden Neuron Result 


Method %R? MSE MAD 
GA-RNN 96.10 0.0027 0.0349 
PSO-RNN 98.47 0.0011 0.0259 
GSA-RNN 65.12 0.0586 0.0837 
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Figure 12. (a) Permeate Flux modeling training result using RNN-PSO five hidden neuron and the residuals 
between actual and model (b) TMP modeling training result using RNN-PSO five hidden neuron and the 
residuals between actual and model 


Table 4. (a) Permeate Flux Modeling Testing Result 


Method AR? MSE MAD 
GA-RNN 93.22 0.0066 0.0468 
PSO-RNN 94.91 0.0049 0.0341 
GSA-RNN 74.47 0.0248 0.0995 


Table 4. (b) TMP Modeling Testing Result 


Method %R? MSE MAD 
GA-RNN 96.68 0.0020 0.0322 
PSO-RNN 98.46 9.4640e-004 0.0243 
GSA-RNN 60.43 0.0620 0.0818 
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Figure 13. (a) Permeate Flux modeling testing result using RNN-PSO three hidden neuron and the residuals 
between actual and model (b): TMP modeling testing result using RNN-PSO three hidden neuron and the 
residuals between actual and model 


5. CONCLUSION 

This paper presents the dynamic modeling technique for submerged membrane filtration process 
using RNN with various soft computing techniques. From the result it showed that the training algorithm 
using soft computing technique is able to produce high accuracy of RNN model. This work compares 
optimization algorithms of GA, PSO and GSA method to train the RNN model. It is observed that the PSO 
algorithm is able to gives higher accuracy compared with the others followed with GA and GSA. The 


Neural Network Model Development with Soft Computing Techniques for ... (Norhaliza Abdul) 


2622 O ISSN: 2088-8708 


performance of the PSO and GA algorithm are almost similar especially for the TMP model. Increment from 
three to five hidden neuron can increase the accuracy of the model, however, it is can also be troublesome for 
GSA algorithm to find an optimum weights and biases value for the model. Finally this model will be useful 
in order to do prediction performance and controller design strategy in the future. 
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